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ABSTRACT 

Let  A  be  a  hermitian  matrix  of  order  n,  and  b  a  known  vector 

n  ^ 

in  C  .  The  problem  is  to  determine  which  vectors  make  =  (x-b)  A(x-b) 

H 

a  maximum  or  minimum  on  the  unit  sphere  U  =  { x  :  x  x  =  1}  .  The  problem 
is  reduced  to  the  determination  of  a  finite  point  set,  the  spectrum  of 
(A,b).  The  theory  reduces  to  the  usual  theory  of  hermitian  forms  when  b  =  0. 
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1.  The  problem. 

i 

I 

Let  A  be  a  hermltiun  square  matrix  of  complex  elements  and  order  n. 

Let  b  be  a  known  n-vector  of  complex  numbers.  For  each  complex  n-vector 
X,  the  nonhomo^cneous  quadratic  expression 

(1.1)  'I'(x)  =  (x-b)^A(x-b) 

(H  denotes  complex  conjugate  transpose)  is  a  real  number.  The  problem, 
suggested  to  us  by  C.  R.  Rao  of  the  Indian  Statistical  Institute,  Calcutta, 
is  to  maximize  (or  minimize)  ♦(x)  for  complex  x  on  the  unit  sphere 
S  =  (x:  X  x  =  1).  Since  ♦  is  a  continuous  function  on  the  compact  set  S, 
such  maxima  and  minima  always  exist. 

In  summary,  our  problem  is: 

(1.2)  maximize  or  minimize  ®(x)  subject  to  x  x  *=  1  . 

The  purpose  of  this  note  is  to  reduce  the  problem  (1.2)  to  the  deter¬ 
mination  of  a  certain  finite  real  point  set  which  we  shall  call  the  spectrum 
of  the  system  (A,b)  (defined  at  end  of  Sec.  l),  and  show  that  a  unique 
number  \  in  the  spectrum  determines  the  one  or  more  x  =  x^  which  maximize 
't>(x)  for  given  b.  Theorem  (^.l)  is  the  main  result.  The  development  is  an 
extension  to  general  b  of  the  familiar  theory  for  the  homogeneous  case  when 
b  =  0,  the  zero  vector.  No  consideration  to  a  practical  computer  algorithm 
is  given  here. 

In  Sec.  7  we  show  that  determining  the  least-squares  solution  of  an 
arbitrary  system  of  linear  equations  Cy  =  f,  subject  to  the  quadratic 


1 


constraint  y  y  “  is  a  special  case  of  problem  (1.2). 

Phillips  (9*2)  and  Twoiney  (9*3)  begin  the  actual  numerical  solution 
of  certain  Integral  equations  by  approximating  them  vith  algebraic  problems 
very  closely  related  to  the  minimum  problem  (1.2). 

Let  5  ^2  -  ’  ‘  -  ^n  (necessarily  real)  eigenvalues  of  k, 

and  let  {Uj^,...,u^}  be  a  corresponding  real  orthonormal  set  of  eigenvectors, 
with  Au ,  “  \ u .  ( 1  *1 ,.  •  .  ,n). 

1  X  1 

Let  a  given  b  be  written 


(1.3) 


n 


( I .  U)  Theorem.  If  x  is  any  vector  in  S  for  whaCh  'I’(x)  is  stationary 
with  respect  to  S,  then  there  exists  a  real  number  \  =  X.(x)  such  that 

(1.5)  A(x-b)  =  X.X  , 

(1.6)  x^x  =  1  . 

Conversely,  if  any  real  \  and  vector  x  satisfy  (l.^,  1.6),  then 
X  renders  't'(x)  stationary. 

Proof ■  Let  x^  be  a  point  of  S.  Now,  as  shown  in  lemma  (8.7), 

t'(x)  is  stationary  at  x^  with  respect  to  x  in  S,  if  and  only  if  there 

H  H 

exists  a  real  Lagrange  multiplier  X.  sn^h  tliat  'i'(x)  =  (x-b)  A(x-b)  \  x  x 

is  stationary  at  x^  with  respect  to  all  neighboring  complex  vectors  x. 
Since 

0  =  ^  grad  4f(x^)  =  A(x^-b)  -  , 

the  theorem  is  proved. 
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To  see  what  conditions  are  satisfied  by  the  \  of  t'heor'-*m  (1.4),  we 
note  that  the  system  (l.5i  1.6)  is  equivalent  to  the  system 


(1.7) 

(1.8) 


(A-\I)x  =  Ab  , 


"  -I 

X  X  =  1 


Let 


n 


X  = 


Then  (l.?)  is  equivalent  to 

n  n 

(1.9)  J]  (^.->^)x.u  =  ^<8  u.  . 

i_l  J.  X  i.  X  i  A 

Def Inition.  By  the  spectrum  of  the  pair  (A,b)  we  mean  the  set  of  all 
real  \  for  which  there  exists  an  x  such  that  (l.T)  and  (1.8)  are  satisfied. 

Given  any  x  satisfying  (l.T)  and  (1.8),  we  shall  say  that  x 
belongs  to  and  frequently  write  x  in  the  form  x^. 

Note  that  the  spectrum  of  (A,0)  is  the  ordinary  spectrum  (X.^)  of 
A. 


2.  Special  case:  no 

Assume  for  the  present  section  that  ^  ^  (all  i).  This  implies 
that  all  /  0,  i.e.,  that  A  is  nonsingular.  If  \  is  in  the  spectrum 
of  (A,b),  (1.9)  implies  that  X,  for  all  i,  and  also  that 


I 
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(2.1) 


■  X..-\ 


( ) 


Then  the  requirement  that 


(2.2) 


c  X  =  lx. 


=  1 


is  equivalent  to  the  condition 


(2-3) 


^  *■!  I*’! 

6('-)  =  I  =  1- 


i=l 


Although  all  X  corresponding  to  stationary  values  of  ^(x)  are 

known  by  theorem  (l.^)  to  be  real,  it  is  useful  to  define  s(X)  by  (2.3) 

for  all  complex  X  not  in  (X^). 

Let  G  be  the  set  of  complex  numbers  X  such  that  g(X)  =  1. 
n 

For  small  enough  ^  is  the  union  of  n  simple  closed  curves  in 

i=l 

thr  complex  plane,  the  k-th  of  which  surrounus  X^^.  As  the  |  |  grow, 

adjacent  curves  first  coalesce  in  double  points,  and  then  merge  into  single 
curves.  For  very  large  values  of  all  jb^],  G  is  one  simple  closed  curve 

irsuluding  all  (X^)  in  its  interior.  The  family  of  sets  G  resembles  the 

_  n 

family  of  lenuiiscates  n  I  X-X^ I  =  const. 

i=l  ^ 

Note,  moreover,  that  g(x)  >  1  for  X  in.slde  any  component  curve 

G  of  G,  while  (,(X)  <  1  in  the  exterior  of  all  components  G.  of  G. 
j  J 

Now  ve  shal?  show  for  the  special  case  of  Sec.  2  that  each  \  In 
G  determines  a  unique  which  satisfies  0.7,  1.8).  For  that 


U 


where  we  define  f  by 


(2-5)  f(^)  =  i>.r  E  —  — -g  • 

1=1 

Fix  and  drop  the  superscript  X  on  x.  To  prove  (2.4),  note  that 

(l.T)  says  (X^-X.)x^  =  Thus 


(\^-X)(x^-b^)  =  X^b^  -  b^(X^-X) 


=  Xb, 


Hence 


f 


and 


n 


'^(x)  =  2  Xj)x^-b^l 


i=l 


Ix^-xl^ 


=  , 

proving  (2.4). 

Since  the  Lagrange  multipliers  X  must  be  real,  the  spectrum  of 
(A,b)  is  the  intersection  of  G  with  the  real  axis.  This  consists  of  from 


% 

‘t 


2  to  2n  distinct  real  numbers.  Hov  many  numbers  are  actually  In  the 
spectrum  depends  on  b;  this  will  be  discussed  in  Sec.  5  for  n  =  2. 

We  wish  to  determine  which  k  in  the  spectrum  corresponds  to  the 
maxjmum  (minimum]  value  of  f(X).  Let  Oj  be  any  component  curve  of  the  set 

G. 

(2.6)  Theorem.  The  maximum  and  mlniinum  real  parts  of  k,  for  X  in  any 
one  Gj ,  both  occur  for  X  on  the  real  axis. 

Proof .  Let  X  =  a  +  if,  with  a,T  real.  Then 


|2 

n  X  |b  1 

g(>^)  =  =  y  - - - ? 

^  A  (a-Xjf  . 


Hence,  for  t  >  0  and  fixed  a,  g(X)  strictly  decreases  as  t  increases. 
Then  in  the  upper  half  plane  t  >  0,  any  line  a  =  constant  intersects  Gj 
in  exactly  one  point.  The  theorem  follows  from  this. 

Def  initlon.  Le\.  denote  the  unique  real  value  of  X  of 

':.aximum  (minimum]  real  part  in  the  set  G. 

T )  Theorem.  Under  the  assumptions  that  A  is  regular  (i^-e.,  X^  ^  0 
for  a] ''  i  )  and  /  0  ( i=l,2, . . .  ,n),  for  all  X  _in  G  such  that 

'  •'‘R’  ^ 

f(Aj^)  <  f(X)  <  f{Aj,)  . 

2  2 

Proof.  Let  =  X^  |b^l  (i=l,...,n).  Introduce  two  independent 
complex  variables  X,|j,  where  p  will  later  be  set  equal  to  X.  In  order 
to  study  the  gradients  of  the  functions  g,  f,  and  h  (defined  below)  for 


A.  we  have 
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complex  ve  shall  use  the  tools  of  Sec.  8.  This  requires  extending 
these  functions  Into  the  space  of  X  and  |j. 

Let  i  =  a  +  IT  (u,T  real).  For  all  complex  \  f  define  the 

functions  and  by 


where 


g(>‘)  =  =  g2(^»^)  > 

n  a^ 


(This  definition  is  consistent  with  (2. 3)*)  Then 


dgA  ragg 

3rj '  [v 


_  ,  by  lemma  (8.1) 
•  u=X 


(2.8) 


(X^-X)(X^.\)^ 


n  a^ 

i?L  lX^-Xl^-(X^-\) 


For  all  complex  X  define  f(X)  by  (2.5).  We  then  define 

the  functions  f^  and  f^  by 

f(X)  =  f^(a,T)  =  f2(X,X)  , 


where 


T 


Then 


1 

2 


by  lemma  (8.1) 


i. .  e. 


(2.9) 


n 

i  kk  ' 

i 

i 

n 

L 

i=l 

k  k^ 

n 

V  _ 

^  1 

k  ■ 

L 

i=l  K 

OJ _ 

1 

•H 

•H 

j.  ~ 

i-^- 

n 

r  _ 

K  , 

■ 

by  (2.8) 

f  1 

l^rj 

_  =  \  • 
^=k 

’^«2' 

u=\ 

rN2 


While  it  if  possible  to  use  (2.9)  to  study  the  behavior  of  f(X.)  on 
the  set  G  where  g(^)  -  1  =  0,  it  is  more  convenient  here  and  in  Sec. 

3  to  introduce  a  new  function  h(\),  which  agrees  with  f(X.)  on  G.  For 
all  complex  k  f  define 


8 


(2.10) 


and  note 

(2.11) 


As 


where 


Then 


(2.12) 


Hence 


(2.13) 


k. 


h(X)  =  f(k)  +  ^  [l-g(\)]  , 


that 


h(X)  =  f(X),  for  \  e  G  . 


with  f  and  g,  we  introduce  functions  h^  and  hg  so  that 


h(X)  =  h^(a,T)  =  , 


dh  df 

3r  " 


2  ^ 


2  on  " 


2Ll-g2(^#M  )] 


by  (2.9) 


x-x  ^82 

=  —  I3r 


ti  (\ 

^T\sr 


+ 


dgA 

srj 


I  ,  by  (8.1) 
Ju=X 

’  [l-g2(X,X)] 

+  i(i-g(x)]  . 
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Now  any  component  Gj  of  the  set  G  where  g(X)  =  1  encloses  a 
region  where  g(X.)  >  1.  On  G  the  gradient  vector  of  g, 


^g-l 

35"  ^ 


is  non-zero,  is  normal  to  Gj,  and  points  to  the  Interior  of  Gj.  Then, 
by  (2.12),  the  gradient  vector  of  h  on  Gj,  namely 


dh,  dh 


dh  /dg  dg\ 

3r  =  ^^35"  ^  ^  3rJ  > 


is  non-zero  for  i  /  0  and  points  along  the  tangent  to  Gj  in  the  direction 
of  increasing  a.  Hence 


(2.1U) 


h(X)  is  strictly  increasing,  as  X  traces  Gj  in 
the  direction  of  increasing  a. 


From  (2.l4)  it  follows  that  h(x)  assumes  its  maximum  veulue,  for  each 


separate  component  curve  Gj  of  G,  at  the  point  Pj  on  Gj  of  maximum 
real  part.  By  theorem  (2.6),  Pj  is  on  the  axis  of  real  X. 

Note  that  setting  =  X  =  X  in  (2.12)  yields  the  result  that 


(2.15) 


h'(X)  =  1  -  g(x),  for  real  X 


To  complete  the  proof  of  the  present  theorem,  we  much  show  that  f(X) 
is  larger  at  the  point  of  xeast  real  part  on  the  component  Gj  of  G 
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than  it  is  at  the  right-most  point  ^  of  the  component  ^ 
immediately  to  the  left  of  G^. 

Note  that  g  is  continuous  for  X.  C  [Pj  and  that 

g(Pj_^)  =  s(aj)  =  ^  tut  g{\)  <  1  for  <  X.  <  Cij.  Then 


pOt 

h(a  )  =  hO  1  ^  h'(\)d\ 

J- 


=  hO^_l)  J  ^  [l-g(>^)ldX  ,  by  (2.15) 


J-l 


>  since  g(X)  <  1  . 

Thus 

(2.16)  ^  ' 

as  was  to  be  proved. 

We  conclude  that  h(\)  increases,  as  X,  increases  along  the  real 
axis  between  adjacent  components  of  G.  Since  h(X,)  =  f(X.)  on  G,  we 
see  from  (2.1^)  and  (2.l6)  that 


majc 
\  €  G 


f(\)  =  f(Ap) 


min 
X.  €  G 


f(\)  =  f(A^) 
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It  follovs  trivially  from  theorem  (2.7)  that  the  maximum  and  minimum 
values  of  f(^.)  over  tne  real  numbers  in  G  (i.e.,  over  the  spectrum  of 


(A,b)  in  the  present  case)  are  also  ^’(^)  respectively. 

By  (1.9)^  our  condition  that  no  X.^b^  =  0  implies  that  \  ^  for 

all  i  and  for  all  X.  in  G.  Hence  and  are  not  eigenvalues  of 

A,  and  so  neither  A  -  A^I  nor  A  -  A^I  is  a  singular  mati'ix.  Therefore 

we  can  solve  equation  (l.?)  uniquely  for  x  and  x  ,  : 

^  max  min 


max 


=  x''^  =  )'^Ab  , 


’‘min  “  =  (A-V)''Ab 


These  equations  give  unique  solutions  to  the  problem  of  minimizing  and 

PI 

maximizing  *I'(x)  =  (x-b)  A(x-b),  for  nonsingular  A  and  b  such  that 
no  b^  =  0. 

It  would  be  desirable  to  be  able  to  prove  that  h(a.)  <  h(p,),  in 

i  J 

the  notation  of  theorem  (2 . 7),  without  analyzing  h(\)  and  g{>~)  for  complex 
values  of 


3-  General  case:  Some 

We  now  study  the  general  case  where  one  or  more  -  0.  To  be 

explicit,  let  ^  =  {a  :  a  set  of  integers.  We  wish  to  examine 

the  spectrum  of  (A,b). 

Define  as  the  set  a  ^  (^  ]  • 
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First,  given  one  a  €  if  ^  ^  is  in  the  spectrum 

of  (A,b)  with  corresponding  vector  x^,  then  (1.9)  shews  that  =  0. 
Thus,  il  \  is  in  the  spectrum  but  not  in  the  set  then  ^  =  0 

for  all  a  €  (^  and.  Just  as  in  the  derivation  of  (2.3),  X.  will  satisfy 
the  equation 


(3.1) 


Conversely,  any  real  solution  k  of  (3-1)  which  is  rot  in  the  set 
will  be  in  the  spectrum  of  (A,b},  and  its  corresponding  vector  will 

have  ~  ^  for  a]],  a  €  (T  .  If  we  interpret  0/0  in  (3.1)  as  0, 
then  it  is  possible  that  some  eigenvalue  in  JC  will  also  satisfy  (3«l'. 

If  so,  we  will  show  that  this  is  also  in  the  spectrum  of  (A,b).  How¬ 
ever,  the  spectrum  may  also  contain  eigenvalues  in  JC  which  do  not 

satisfy  (3.l)>  as  we  shall  now  show.  No  eigenvalue  X.j^  not  in  ^  can 
be  in  the  spectrum,  because  ^  ~  would  make  the  left  side  of  (3*1) 
equal  to  oo  . 

Fix  attention  on  one  for  k  a  C  >  We  wish  to  examine  the 

possibility  that  this  is  in  the  spectrum  of  (A,b).  Let  m  be  the 

m  iluiplic  ity  of  \  as  an  eigenvalue  of  A.  Let  =  (i:  X..  =  X,,  )  so 

^  K  K  1  K 

that  card  =  m.  If  k^  is  in  the  spectrum,  then  (1.9)  shows  that 

X^b^  -  u  i  or  all  Q  in  Moreover,  if  k^  is  in  the  spectrum,  then 

the  corresponding  vector  x  =  x^k  has  the  properties 


X  .  =  T - — 

1  X  -X, 


(i  <  , 
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and,  by  (2.1),  also 


(3-2) 


Conversely,  if  (3*2)  holds  then  we  can  always  define  for  all 


;  C  in  such  a  way 


that 


(3-3) 


Hence,  by  (3*2)  and  (3*3)»  equation  (1.8)  holds  and,  since  (l.T)  is 
satisfied,  is  in  the  spectrum  of  (A,b). 

If  equality  holds  in  (,3.2)  then  x.  must  be  0  for  all  i  € 
i.e.,  satisfies  (3*1))  and  x  is  unique.  But  if  Inequality  < 

holds  in  (3-2),  then  there  is  an  (m-1  )-dimensional  sphere  'iJ  of  values 

jfN 

of  (x^),  for  i  e  which  satisfy  (3-3)'  For,  if  a  point 

(x^  »---^x^  )  is  in  U,  then  so  are  all  points  of  form 
1  m 


10, 


iO 


X,  *3 

1  m 


^  o  '^)  (all  0^  real)  , 


E  ' 


since  «  l^j^l  1^  constant  for  all  of  these.  In  this  case 


uniqueness  of  x  is  lost.  The  sphere  is  analogous  to  (ir  fact  is  a 
generalization  of)  the  sphere  of  unit  eigenvectors  of  a  hermitian  matrix 
A  belonging  to  an  eigenvalue  of  multiplicity  m  . 


Ik 


Note  that  an  inequality  <  in  (3*2)  states  that  \  is  in  the 
exterior  of  the  graph 


G 


i.e.,  can  be  Joined  to  «  by  an  arc  not  cutting  G.  Thus,  in  brief, 

the  spectrum  of  (A,b)  consists  of  the  union  of  all  real  numbers  in  the  set 


(3.M 


where  we  interpret  0/0  as  0,  with  those  numbers  which  are  exterior 

to  the  graph  G.  (If  G  is  the  null  set,  then  b  =  0  and  the  spectrum 
of  (A,0)  consists  of  all  eigenvalues 

We  must  now  examine  ’t’(x^)  for  \  in  the  spectrum  of  (A,b).  The 
study  of  'I’(x^),  for  real  \  c  G  in  (3*^)  is  the  same  as  in  Sec.  2, 
and  yields  the  same  results  (2.4)  and  (2. 5):  First,  for  X.  £  G, 

X  ? 

)  =  f(4)  -  |\1  52  - —  ,  where  O/O  =  0.  Second,  let  iuq, 

1%  « 

be  the  right-most  [resp.  left-most]  points  of  G.  Then  f(^n)  maximizes 

ij  n 

[resp.  f(M,  )  minimizes]  f(X.)  for  X.  £  G.  It  remains  to  consider 

<I>(x  ),  lor  eigenvalues  outside  G. 

K 
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(j.^)  Theorem.  For  any  X  In  the  spectrum  of  (A,b)  ve  have 


(3.6) 


®(x^)  =  h(X)  =  f(X)  +  X[l-g(X)]  , 


where  f(X)  is  given  by  (2.3),  with  O/O  Intepreted  as  0. 


Proof.  Take  any  X  in  the  spectrum  of  (A,b). 


If  \  (k=l,...,n),  then  X  €  G,  and  everything  proceeds  as 

in  the  proof  of  (2.i+),  showing  that  ®(x^)  =  f(X).  Since  g(X)  =  1,  we 
have  proved  (3*6)  when  X  /  Xj^. 

If  X  =  X,  ,  an  eigenvalue  of  A,  let  x.  denote  the  i-th  coordinate 
^k 

of  any  x  which  satisfies  (I.5)  and  (I.6)  (and  hence  (3*2)  and  (3*3))' 
Since  Xj^  is  in  the  spectrum  of  (A,b),  we  have  =  0  for  all 

i  €  where  is  defined  above  after  (3*l)>  and  as 

Xilxi-bil^  =  X^lxJ^  =  i  ^  ®k- 


(3.7) 


.  h, 


=  X, 


1  - 


=  X 


k 


n  X,  lb. 

1  -  Z  ' 


1=1 


hi'’! 


i2  1 


,  where  O/O  =  0 


Moreo.er,  like  {2.k)  we  can  prove 
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(3.8) 


E  \  E  —  - 1 


P  n  »•,  |b,  I 

=  X,  5]  - 5  ,  where  O/O  =  0 

1=1 


Adding  (3*7)  to  (3*8),  we  get 


1=1  ^  ^ 


(3-9) 


.2  ^  ^  1^1  .  .  r  ^  "l  l"il 

^  O  ^  ^k  ^  ”  2j  ? 


f{\)  + 


=  h(Xj^)  . 


This  proves  (3*6)  when  X  =  Xj^. 

It  Is  property  (3*6)  of  h  which  motivated  our  use  of  h  In  Sec.  2. 


Note,  It  is  easily  shown  from  (3.6)  or  (3.9)  that,  for  all  X  in 
the  spectrum  of  (A,b), 


(3.10) 


n  \  jb 

h(k)  .  k  +  k  E  k-k 


itTl  ’ 


where  O/O  ■  0  . 


If  X  Is  in  the  spectrum  of  (A,b),  but  is  not  an  elgenv^ue  of  A 
we  can  derive  (3>10)  as  follows.  Let  x  belong  to  X.  Oien 


<l>(x)  =  (x-b)^A(x-b) 

=  (x-b)”xx  ,  by  (1.5) 
=  X  x^x  -  X  b^^x 


We  shall  not  make  use  of  (3J.0)  here. 

We  now  usf'  formula  (3«6)  to  extend  the  domain  of  h  to  all  real  X 
where  g(X)  <  •,  i.e.,  to  all  X  except  where,  for  some  1,  X  =  X^ 
and  /  0* 

Ae  stated  before  (3*5)»  ve  know  that  the  largest  value  of  =  h(X) 

for  X  in  G  occurs  at  the  right-most  point  u_  uf  G.  It  remains  to 

n 

see  whether  h(Xj^)  may  be  still  larger  for  any  Xj^  in  the  tpectrum  of 
(A,t),  if  Up  <  Xj^. 

The  answer  is  furnished  by  formula  (2.15),  which  is  valid  for  the 
general  case  of  Sec.  3  with  the  understanding  that  O/O  =  0.  Thus  h  is 
increasing  on  all  segments  of  the  real  axis  between  or  exterior  to  components 
of  the  curve  G.  It  follows  that  h(X)  takes  its  maximum  at  the  rightmost 
point  Ap  of  the  spectrum  of  (A,b)  and  its  minimum  value  at  the  left¬ 
most  point  of  the  spectrum  of  (A,b),  whether  or  not  these  are 

eigenvaluee  of  A. 


18 


From  the  considerations  following  (3*3)>  we  see  that  the  maximizing 
X  is  unique  if  €  G.  If 

eigenvalue  of  A  of  multiplicity  m,  then  the  maximizing  x  include  all 
points  of  un  (m-l)-sphere  of  nonzero  radius,  whose  center  is  not  at  0 
when  b  ^  0. 

The  above  result  about  and  A.  for  the  case  where  some  A..b.  -  0 

R  L  i  1 

can  be  obtained  by  continuity  from  the  case  where  no  =0.  It  is  not 

clear  that  we  could  use  continuity  to  deduce  the  nature  of  the  maximizing 
and  minimizing  vectors,  for  multiple  roots. 


however,  is  not  in  G  and  is  an 


k.  The  main  result. 


In  Secs.  2  and  3  we  have  proved  our  result: 

(^.l)  Theorem.  Given  A,  hermit ian  with  eigenvalues  (X,^),  and  b, 
arbitrary,  define  (b^)  as  in  (1.3)-  Then  the  spectrum  of  (A,b) 
consists  of  all  real  \  such  that 

n  jb.]^ 

i-1 

'  1  ‘ 


together  with  each  eigenvalue  V  of  A  for  which  g(X-,. )  <  1. 


For  each  \  in  the  spectrum  with  g(X.)  -  1,  a  unique  x^  is 
found  by  solving  (l.T,  1.8).  For  each  k  in  the  spectrum  with  g(X.)  <  1, 
there  exists  an  (m-l)-sphere  of  x^  satisfying  (l.T>  1.8);  where 


ra  =  card 


✓ 
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Each 


so  found  renders  ®(x)  stationary  on  S.  Let 


/'.^=  niax(X.  :  A,  e  spectrum);  let  -  min{X.  :  \  £  spectrum).  Let 

\  \  “  ^ 

=  any  x  ;  let  x _  =  any  x  .  Then  minimizes  ^(x)  on  S, 

ni£Lx  — min 

and  ®(x  )  maximizes  <l’(x)  on  S. 

-  '  max  -  — 


5 .  The  number  of  points  in  the  spectrum. 

As  ve  noted  in  Sec.  2,  if  A  is  of  order  n,  then  the  spectrum  of 
(A,h)  contains  anywhere  from  2  to  2n  real  numbers.  When  does  it  have 
the  full  amount  2n?  If  any  =  0,  then  the  discussion  of  Sec.  3 

showed  that  the  spectrum  necessarily  has  fewer  than  2n  points.  So  we 
are  limited  to  the  case  where  all  X.^b^  ^  0.  But  then,  as  shown  in  Sec. 

2,  we  know  that  the  spectrum  is  the  intersection  of  the  graph  of 


for  real  \  with  the  line  =  1. 

The  graph  of  (5*1)  real  X  consists  of  n  +  1  branches  between 

the  n  vertical  asymptotes  X  =  X,  (i=l,...,n).  Since  u  >  0  for  all 

X,  and  n  -♦  0  as  X  -*  «  and  X  the  right-most  and  left-most 

branches  necessarily  cut  n  =  1.  The  spectrum  has  the  full  number  2n  of 

points  if  and  only  if  each  of  the  n  -  2  interior  branches  of  the  curve 

reaches  its  minimum  with  u  <  1.  For  general  n  a  condition  for  this  is 
probably  too  complicated  to  derive.  For  n  -  2,  however,  we  can  answer 
the  question,  as  follows: 
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( 5-2)  Theorem. 


(5-3)  Let  n  =  2,  and  assume  A  Is  in  diagonal  form  vlth  . 

If  the  spectrum  of  (A,b)  consists  of  U  distinct  numbers,  then 


(5.^) 
and  also 


(5.5) 


0  <  lb^X.^1  and  0  <  lb2X.2l  , 


(5-6)  Conversely^  if  (5*^)  and  (5-5)  hold^  then  the  spectrum  of 
(A,b)  consists  of  U  distinct  numbers. 

Pi-QOf  of  (5.3).  Let  a^  =  |\^b^|^  (1=1,2).  If  either  a^  or  a^ 

were  zero,  then  the  development  in  Sec.  3  shows  that  the  spectrum  would  con 
sists  of  at  most  3  poin.''-.  Hence  a^  >  0  and  a^  >0;  i.e.,  (5*^)  holds. 
Let  M  =  (a2/a^)^'^^.  Now  the  development  in  Sec.  2  shows  that  the 
spectrum  of  (A,b)  consists  precisely  of  the  real  roots  X.  of  the  equation 

(5.7)  g(^)  =  - ^  +  - ^2"’  • 

(\.\^)^  (\-\^r 


Since  (5-7)  has  4  real  roots,  we  know  that  two  roots  must  lie  in  the  internal 
(X^,X.2).  Now  let  u  be  the  unique  real  root  ol' 


g'(^) 


-2a, 


2a, 


0 
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Then,  because  there  are  two  roots  of  (5*T)  In  (X.,X.p), 


(i-8) 


g(M)  <  1  • 


We  now  show  that  (5 -8)  Implies  (5*^)* 
Solving  g'(n)  =  0  shows  that 


A. 


whence 


Hence 


g(^i)  = 


“IT  = 

> 

a^(l+M)^ 

A 

M^(: 

^1  ■ 

2 

(1+M)^  a^ 

1 

^1 

o 

[4  *  4) 

c. 

(  ^ 

|-3 

1 

^2 


1  1 

^  -  a^ 
^2 
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I 


Thus  g(^)  <  1  implies 

11  2 

:5.9)  ®1  ^  ' 

vhich  implies  (5*5  )•  Thus  (5*3)  is  proved. 

Proof  of  ( ^ . 6 ) .  We  have  >  0,  >  0,  and  (5-9)-  The  above  steps 

are  reversable,  and  so  g(u  )  <  1,  whence  there  are  k  real  roots  of  g(u  )  =  1. 
Thus  theorem  (5*2)  is  completely  proved. 

Condition  (5-^)  says  that  neither  nor  is  0,  and  that  the 

point  b  =  does  not  lie  on  an  axis  of  the  )- plane .  Condition 

(5.5)  requires  that  (b^jb^)  be  inside  a  curve  P  which  depends  only  on 
the  ratio  \^/\^.  If  ^2^^!  “  example,  the  curve  P  is 

ibj2/3. 


I 


In  Figure  1  the  number  of  points  in  the  spectrum  of  (A,b)  is 
indicated  for  different  b  in  the  first  quadrant  by  integers  in  circles. 
If  Ko/^1  1  >  2,  the  curve  F  includes  values  Ib^  [  >  1.  But 
^  ^  implies  thai,  on  F,  [b^|  <  1. 


6.  Geometrical  interpretation. 

The  surfaces  'I’(x)  -  k  are  similar  conic  surfaces  with  center  b  in 
the  euclidean  n-space  of  vectors  x.  The  maximum  problem  (1.2)  is  to 

find  the  conic  surface  with  maximum  k  which  touches  the  constraint  sur¬ 
face  S,  the  unit  sphere  in  The  rotation  of  A  to  diagonal  form  is 

a  rotation  of  (leaving  S  invariant,  of  course)  which  causes  principal 

axes  of  the  conic  surfaces  to  coincide  with  the  axes  of  ^  . 

n 

The  vector  Ax  -  b  is  half  the  gradient  of  *I*(x),  and  x  is  the 
radius  vector.  Condition  (1.5)  merely  states  that  at  a  point  where  'l’(x) 
is  stationary,  for  x  on  S,  the  surface  't'(x)  =  k  is  tangent  to  S. 

Fix  x  at  a  solution  of  (I.5),  and  let  t  be  real.  If  the  constant 
^  of  (1.5)  is  positive,  the  value  of  'I>(tx)  increases  as  t  Increases 
from  1;  if  \  is  negative,  't'(tx)  decreases  as  t  increases  from  1. 

The  main  result  of  Secs.  2  and  3  is  that  the  maximum  problem  of  Sec. 

1  is  solved  for  the  largest  value  of  K  satisfying  (1.5),  fo”  x  on  S. 

The  authors  see  no  obvious  geometrical  reason  why  this  should  be  so. 

If  all  /  0,  then  Sec.  2  shows  that  any  vector  x  =  x'^  which 

makes  ®(x)  stationary  on  S  is  uniquely  determined  by  k. 

Figure  2  shows,  for  n  -  2  and  0  <  k^^  <  k^,  a  case  where  there  are 


2h 


k  distinct  points  of  tangency  of  an  ellipse  with  the  unit  circle.  All 
ellipses  have  center  b  and  common  value  of  ^2/^1  ^  Since 


Figure  2 


^g/^l  >2,  it  was  shown  in  Sec.  5  that  k  distinct  tangencies  were  possible 
for  certain  b  outside  S. 

Whenever  some  bj^  =  0,  then,  provided  that  (3*2)  holds  with  the 
inequality  sign  <,  we  get  more  than  one  x  belonging  to  a  given  X..  That 
is  illustrated  in  Fig.  3;  where  n  =  2  and  k  =  1.  What  is  not  obvious  to 
the  authors  is  a  geometrical  reason  why  necessarily  ^  this  case. 

X 

b 


another  x 

Figure  3 
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7*  A  constrained  least  squares  problem. 


Let  C  be  an  m  x  n  matrix  (m  >  n)  and  f  an  m-vector,  both 

over  the  complex  field.  We  wish  to  study  the  set  of  complex  n-vectors  y 

H  l/? 

of  euclidean  length  ||y|l  =  (yy)^  =l  such  that 
(7.1)  llCy-flP  =  (Cy-f)“(Cy-f )  =  min. 


The  constraint  is 


(7.2) 


Because  euclidean  length  is  invariant  under  unitary  transformations, 
it  is  useful  to  rotate  coordinates  in  both  the  space  of  y  and  the  space 
of  f.  To  do  this,  let  r  =  rank(C),  and  write 

(7.3)  c  =  u“dv  , 

where  U,  V  are  unitary,  and  vhere  the  only  non-zero  elements  of  D  are 
the  first  r  elements  of  the  leading  diagonal,  which  we  may  arrange  so 
that 


d,  >  do  >  •  •  •  >  d  >  0 

1  —  —  —  r 


Now  let  V;-  =  x  and  Uf  =  g.  Then 
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Cy  .  f  =  U^DVy  -  lAjf 
=  U^(Dx-g)  . 


Hence 


(7.^) 


||Cy-f|p  =  ||Dx-gll^ 


=  E  Id^x  -g  1^  >*■ 
1=1  ^  ^ 


m 

E 

i=r+l 


The  problem  (7.1,  7*2)  is  to  minimize 


^  i2  f  ,2 


E  IVi'^ll  "  ^  "^i  l''i-8i/‘^ir 

1=1  1  i  1  1=1  ^ 


subject  to  the  constraint 


n 


(7-5) 


E  l^^il  =  1  • 

i=l 


Nov  let 


and  let 


0  (i  =  1,2, . . . ,n-r) 


(  n+l-i  (i  =  n-r+l,...,n) 
(i  =  l,2,...,n-r) 


^n-t-l-i 
'  ‘^n+l-l 


( i  =  n-r+1, . . .  ,n)  . 


We  then  have  changed  our  problem  to  one  of  minimizing 
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(7.6) 


} 


2 


subject  to  the  constraint  (T-5)>  where 


(7-7) 


X  <x  <.**<x 

n-r  n-r+1  -  -  n 


This  is  precisely  the  minimum  problem  (1.2)  of  Sec.  1.  The  special  role  of 
the  n  -  r  zero  eigenvalues  of  C  C  becomes  evident. 

Thus  the  general  problem  of  the  least-squares  solution  of  Cy  =  f 
with  constraint  (7.2)  is  a  special  case  of  our  minimum  problem  (1.2). 


8.  Lemmas  from  complex  function  theory. 

In  this  final  section  we  state  and  prove  three  lemmas  relating  partial 
derivatives  of  certain  regular  analytic  functions  of  several  complex 
variables  to  gradients  of  real-valued  functions  of  vector  variables.  This 
technique  is  common  in  the  study  of  second-order  partial  differential 
equations;  for  example,  see  (9*1).  We  include  the  material  mainly  to 
keep  cur  treatment  self-contained,  and  partly  to  call  explicit  attention  to 
the  fact  that  the  Lagrange  multiplier  X  must  be  real  even  though  complex 
variables  are  used. 

(8.-l)  Lemma.  Let  'I’(X,^)  be  a  regular  analytic  function  of  two  complex 
variables  X,  u  such  that,  for  all  real  x,  y. 
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(8.2) 


F(x,y)  =  «(x+iy,  x-ly) 


is  real -valued.  Then 


+ 


i 


bT 


2 


b^ 


X.=x+iy 

^=x-iy 


Proof .  Differentiate  (8.2); 


(8.3) 


dl'  d<l>  T 

'  s:  •  ^  ^ 


(8.U) 


bY  b^  ,  b^  , 

37  ‘  35^  ‘ 


Add  (8.3)  to  (8.U)  X  i: 


dF  ^  ,  dF 
3J  *  i  ^ 


=  2 


dC> 


(8.5)  Lemma.  Let  F  and  G  be  real-valued  differentiable  functions  of  real 

variables  x,  ,yT,...,x  ,y  .  For  abbreviation,  let  z,  =  x,  +  ly,  ,  and  let 
-  l'*'l  n  •^n  * -  k  k  •'k'  - 

z  =  (z^>-.‘^z^)*  Then,  for  F(z)  to  be  stationary  at  z  =  a  vith  respect 
to  all  neighboring  z  such  that  G(z)  =  G(a),  it  is  necessary  and 
sufficient  that  there  exist  a  re^  Lagrange  constant  X.  such  that 


(8.6) 


=  0 


for  z  =  a  and  k  =  1, . . . ,n. 
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Proof .  Condition  (8.6)  is  nothing  but  the  usual  condition  that  the 


real  gradient  vector 


dF  8f  dF  dF 


dF  dF  \ 

37"  '  3r  J 

n  n  / 


be  parallel  to  the  vector 


do  do  do  do 
3)^  ’  ^  ’  3^  ’  ^ 


do  ^  \ 

dx  ’  dy  J 
n  n  / 


The  use  of  tne  complex  variables  is  unessential. 

Given  any  vector  z  =  (z^,...,z^),  we  let  z  denote  the  vector  of 
complex  conjugates  (z^^, . . .  ,z^). 

(8.7 )  Lemma.  Let  ®(z,w)  and  4f(z,w)  be  regular  analytic  functions  of  the 
tvo  complex  vector  variables  z  =  (z^,...,z^)  and  w  =  (v^,...,w^)  with 

the  property  that  ^{z,z)  and  '^{z,z)  are  real.  Tlaen  'l>(z,z)  is 

stationary  at  z  =  a  vith  respect  to  all  z  such  that  ijf(z,z)  =  t(a>a)# 

if  and  only  if  there  exists  a  re^  Lagrange  constant  \  such  that 

d<f  1  n 

k  k 

for  z  =  a  and  w  =  a  and  k  -  l,2,...,n. 

Proof.  Let  z  =  x  +  iy.  Then  ^{z,z)  =  F(x,y),  ^{z,z)  =  G(x,y).  By 
lemma  8.1  applied  to  each  variable  z^, 
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dw, 

k 


dF 


+  i 


dF 


f 


1 


dG 


1  or  z  =  a,  V  -  a,  and  k  = 

Then  lemma  (8.7)  follows  from  lemma  (8.5)  • 
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